State-Conditioned Adversarial Subgoal Generation

Hierarchical reinforcement learning (HRL) proposes to solve difficult tasks by performing decision-making and control at successively higher levels of temporal abstraction. However, off-policy HRL often suffers from the problem a non-stationary high-level policy since low-level is constantly changing. In this paper, we propose novel approach for mitigating non-stationarity adversarially enforcing generate subgoals compatible with current instantiation policy. practice, adversarial implemented training simple state conditioned discriminator network concurrently which determines compatibility level subgoals. Comparison state-of-the-art algorithms shows that our improves both efficiency performance in challenging continuous tasks.